DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 20 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim does not fall within at least one of the four categories of patent eligible subject matter because Claim 20 is directed to a signal per se. Claim 20 recites a computer-readable medium. The broadest reasonable interpretation of a claim drawn to a computer readable medium (also called machine readable medium and other such variations) typically covers forms of non-transitory tangible media and transitory propagating signals per se in view of the ordinary and customary meaning of computer readable media, particularly when the specification is silent. See MPEP 2111.01. When the broadest reasonable interpretation of a claim covers a signal per se, the claim must be rejected under 35 U.S.C. 101 as covering non-statutory subject matter. The USPTO recognizes that applicants may have claims directed to computer readable media that cover signals per se, which the USPTO must reject under 35 U.S.C. 101 as covering both non-statutory subject matter and statutory subject matter. A claim drawn to such a computer readable medium that covers both transitory and non-transitory embodiments may be amended to narrow the claim to cover only statutory embodiments to avoid a rejection under 35 U.S.C. $ 101 by adding the limitation "non-transitory" to the claim. Such an amendment would typically not raise the issue of new matter, even when the specification is silent because the broadest reasonable interpretation relies on the ordinary and customary meaning that includes signals per se.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-2, 6-7, 11-13, 17 and 19-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Gardiner (US 20120057064 A1).
Regarding Claim 1, Gardiner teaches a display method, applied to an electronic device (Paragraph [0006]: “An embodiment of the present invention is a mode of operation in a portable electronic device, for stabilizing the display of the device with respect to the user's face”), and comprising:
Obtaining, when the electronic device is in or enters a landscape side-standing state, a first landscape/portrait direction of the electronic device determined based on a face direction (Paragraph [0016]: “The device's accelerometer or other integrated inertial sensor is monitored at 24 to determine when the device 10 has been tilted sufficiently to otherwise trigger a switch of the display mode from portrait to landscape or vice versa. This may be an entirely conventional operation. If a tilt is detected in such operation, an image of the user's face captured with the front-facing camera at that point is again analyzed at 26 to determine if the user's orientation has likewise changed. If the orientation of the device has tilted without a corresponding change in the user's orientation, then the display mode is switched at 28. On the other hand, if the user's orientation has also changed such that the user's face remains in approximately the same orientation relative to the display screen, then the current display mode is maintained at 30”); and
If the first landscape/portrait direction is a portrait direction, setting a screen display direction of the electronic device to portrait display (Paragraph [0016]: “The device's accelerometer or other integrated inertial sensor is monitored at 24 to determine when the device 10 has been tilted sufficiently to otherwise trigger a switch of the display mode from portrait to landscape or vice versa. This may be an entirely conventional operation. If a tilt is detected in such operation, an image of the user's face captured with the front-facing camera at that point is again analyzed at 26 to determine if the user's orientation has likewise changed. If the orientation of the device has tilted without a corresponding change in the user's orientation, then the display mode is switched at 28. On the other hand, if the user's orientation has also changed such that the user's face remains in approximately the same orientation relative to the display screen, then the current display mode is maintained at 30”. Notes: the display is in a portrait mode or landscape mode dependent on the orientation of the device, and further dependent on the orientation of the face of the user).
Regarding Claim 2, the method according to Claim 1 is rejected over Gardiner.
Gardiner teaches the method according to claim 1, wherein the obtaining a first landscape/portrait direction of the electronic device determined based on a face direction comprises:
obtaining the face direction recognized based on image data collected by a front camera (Paragraph [0015]: “An image of the user's face is captured with the front-facing camera”); and
determining the first landscape/portrait direction based on an included angle a between the face direction and a natural direction of the electronic device (Paragraph [0016]: “if the image feature of the user's face changes orientation with respect to the device's orientation by less than a threshold amount (e.g., a predetermined angle such as 45.degree.), then the switching of the display mode is disabled”),
wherein the natural direction of the electronic device is a direction from a lower end of the electronic device to an upper end, and the upper end is an end at which the front camera and a rear camera are located (Paragraph [0013]: “Portable handheld electronic devices, such as the iPod.TM. and iPhone.TM. multifunction devices by Apple Inc., have a display screen for viewing photos, videos, documents, web content and other media”; Paragraph [0015]: “An image of the user's face is captured with the front-facing camera”. Notes: portable electronic devices like Iphones are well known to be oriented with a lower end and an upper end, where the upper end has a front and rear camera).
Regarding Claim 6, the method according to Claim 2 is rejected over Gardiner.
Gardiner teaches the method according to claim2, further comprising:
Obtaining, when the electronic device is in or enters the landscape side-standing state, the image data collected by the front camera (Paragraph [0015]: “An image of the user's face is captured with the front-facing camera 14 and is then analyzed at 22 to establish a starting reference for the user's orientation”. Notes: The camera is inherently capable of capturing image data regardless of the device position);
Performing face direction recognition based on the image data (Paragraph [0016]: “The device's accelerometer or other integrated inertial sensor is monitored at 24 to determine when the device 10 has been tilted sufficiently to otherwise trigger a switch of the display mode from portrait to landscape or vice versa. This may be an entirely conventional operation. If a tilt is detected in such operation, an image of the user's face captured with the front-facing camera at that point is again analyzed at 26 to determine if the user's orientation has likewise changed. If the orientation of the device has tilted without a corresponding change in the user's orientation, then the display mode is switched at 28. On the other hand, if the user's orientation has also changed such that the user's face remains in approximately the same orientation relative to the display screen, then the current display mode is maintained at 30. In other words, if the image feature of the user's face changes orientation with respect to the device's orientation by less than a threshold amount (e.g., a predetermined angle such as 45.degree.), then the switching of the display mode is disabled ”).
Regarding Claim 7, the method according to Claim 6 is rejected over Gardiner.
Gardiner teaches the method according to Claim 6, wherein the performing face direction recognition based on the image data comprises:
If only one face is recognized based on the image data, performing face direction recognition on the one face (Paragraph [0016]: “Multiple images of the user, which may be captured by the same camera 14 as the device 10 is tilted from one orientation to another (e.g., horizontal to vertical or vice versa), may be analyzed to make the determination as to how much the user's feature changes its orientation”; Paragraph [0015]: “the iPhoto.TM. application by Apple Inc. uses facial detection to identify faces of people in photographs”).
Regarding Claim 11, the method according to Claim 1 is rejected over Gardiner.
Gardiner teaches the method according to Claim 1, further comprising:
when the electronic device exits the landscape side-standing state or is in a landscape non-side-standing state (Paragraph [0016]: “The device's accelerometer or other integrated inertial sensor is monitored at 24 to determine when the device 10 has been tilted sufficiently to otherwise trigger a switch of the display mode from portrait to landscape or vice versa”),
obtaining a second landscape/portrait direction of the electronic device determined based on a gravity direction (Paragraph [0016]: “The device's accelerometer or other integrated inertial sensor is monitored at 24 to determine when the device 10 has been tilted sufficiently to otherwise trigger a switch of the display mode from portrait to landscape or vice versa. If a tilt is detected in such operation, an image of the user's face captured with the front-facing camera at that point is again analyzed at 26 to determine if the user's orientation has likewise changed. If the orientation of the device has tilted without a corresponding change in the user's orientation, then the display mode is switched at 28. On the other hand, if the user's orientation has also changed such that the user's face remains in approximately the same orientation relative to the display screen, then the current display mode is maintained at 30”. Notes: A gravity direction, in its broadest reasonable interpretation, is simply downward, and a second landscape/portrait direction based on a gravity direction is simply deciding whether the user’s orientation affects the normal upright display of a screen); and
determining the screen display direction of the electronic device based on the second landscape/portrait direction (Paragraph [0016]: “The device's accelerometer or other integrated inertial sensor is monitored at 24 to determine when the device 10 has been tilted sufficiently to otherwise trigger a switch of the display mode from portrait to landscape or vice versa. If a tilt is detected in such operation, an image of the user's face captured with the front-facing camera at that point is again analyzed at 26 to determine if the user's orientation has likewise changed. If the orientation of the device has tilted without a corresponding change in the user's orientation, then the display mode is switched at 28. On the other hand, if the user's orientation has also changed such that the user's face remains in approximately the same orientation relative to the display screen, then the current display mode is maintained at 30”).
Regarding Claim 12, the method according to Claim 6 is rejected over Gardiner.
Gardiner teaches the method according to Claim 6, wherein the obtaining, when the electronic device is in or enters a landscape side-standing state, the image data collected by the front camera comprises:
When the electronic device is in or enters the landscape side-standing state, and there is a landscape/portrait auto-rotation requirement in a current scenario, obtaining the image data collected by the front camera (Paragraph [0016]: “The device's accelerometer or other integrated inertial sensor is monitored at 24 to determine when the device 10 has been tilted sufficiently to otherwise trigger a switch of the display mode from portrait to landscape or vice versa. If a tilt is detected in such operation, an image of the user's face captured with the front-facing camera at that point is again analyzed at 26 to determine if the user's orientation has likewise changed”).
Regarding Claim 13, the method according to Claim 6 is rejected over Gardiner.
Gardiner teaches the method according to Claim 6, wherein the image data collected by the front camera is obtained in a real-time sampling manner (Paragraph [0016]: “The device's accelerometer or other integrated inertial sensor is monitored at 24 to determine when the device 10 has been tilted sufficiently to otherwise trigger a switch of the display mode from portrait to landscape or vice versa. If a tilt is detected in such operation, an image of the user's face captured with the front-facing camera at that point is again analyzed at 26 to determine if the user's orientation has likewise changed”).
Regarding Claim 17, the method according to Claim 1 is rejected over Gardiner.
Gardiner teaches determining, on an application processor side (Paragraph [0015]: “A process for implementing an embodiment of the present invention is shown in FIG. 2. The process can be performed by a suitably programmed processor in the device 10 (e.g., an applications processor),
whether it is necessary to fuse the first landscape/portrait direction determined based on the face direction and a second landscape/portrait direction determined based on a gravity direction, to determine the screen display direction of the electronic device (Paragraph [0016]: “The device's accelerometer or other integrated inertial sensor is monitored at 24 to determine when the device 10 has been tilted sufficiently to otherwise trigger a switch of the display mode from portrait to landscape or vice versa. If a tilt is detected in such operation, an image of the user's face captured with the front-facing camera at that point is again analyzed at 26 to determine if the user's orientation has likewise changed. If the orientation of the device has tilted without a corresponding change in the user's orientation, then the display mode is switched at 28. On the other hand, if the user's orientation has also changed such that the user's face remains in approximately the same orientation relative to the display screen, then the current display mode is maintained at 30”).
Regarding Claim 19, the method according to Claim 1 is rejected over Gardiner.
Gardiner teaches the display method according to Claim 1, wherein the display method is performed on an electronic device (Paragraph [0025]: “The invention also relates to apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, or it may comprise a general purpose computer”), comprising: one or more processors; a memory; and one or more computer programs (Paragraph [0025]: “The invention also relates to apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, or it may comprise a general purpose computer selectively activated or reconfigured by a computer program stored in the computer), wherein the one or more computer programs are stored in the memory, and the computer programs are executed by the one or more processors to enable the electronic device to perform the display method according to Claim 1 (Paragraph [0025]: “This apparatus may be specially constructed for the required purposes, or it may comprise a general purpose computer selectively activated or reconfigured by a computer program stored in the computer”).
Regarding Claim 20, the method according to Claim 1 is rejected over Gardiner.
Gardiner teaches the display method according to Claim 1, wherein the display method is performed by a computer program run on an electronic device, wherein a computer-readable storage medium comprises the computer program, enabling the electronic device to perform the display method ((Paragraph [0025]: “The invention also relates to apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, or it may comprise a general purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program is stored on or transmitted on a machine-readable medium”).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 3, 10 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Gardiner (US 20120057064 A1).
Regarding Claim 3, the method according to Claim 2 is rejected over Gardiner.
Gardiner teaches the method according to Claim 2, wherein the determining the first landscape/portrait direction based on an included angle between the face direction and a natural direction of the electronic device comprises:
When an
a
indicating the angle between the face direction and natural direction of the device is within 45 degrees of a portrait orientation, the first landscape/portrait direction is a portrait direction (Paragraph [0016]: “if the image feature of the user's face changes orientation with respect to the device's orientation by less than a threshold amount (e.g., a predetermined angle such as 45.degree.), then the switching of the display mode is disabled”; Refer to Figure 1A, Figure 1B, and Figure 1C. Notes: in other words, if the user angle corresponds with the device angle and stays within 45 degrees of it, then the display does not rotate even if the device is in a landscape mode); and
When an
a
indicating the angle between the face direction and natural direction of the device is within
β
, where
β
indicates a direction recognition range threshold, the first landscape/portrait direction is a portrait direction (Paragraph [0016]: “If the orientation of the device has tilted without a corresponding change in the user's orientation, then the display mode is switched at 28. On the other hand, if the user's orientation has also changed such that the user's face remains in approximately the same orientation relative to the display screen, then the current display mode is maintained at 30. In other words, if the image feature of the user's face changes orientation with respect to the device's orientation by less than a threshold amount (e.g., a predetermined angle such as 45.degree.), then the switching of the display mode is disabled”; Refer to Figure 1A, Figure 1B, and Figure 1C. Notes: in other words, if the user angle corresponds with the device angle and stays within angle threshold of it, then the display does not rotate even if the device is in a landscape mode. If the angle threshold is exceeded, then the display mode is switched (portrait direction displayed at the angle)).
Gardiner does not explicitly teach When a
a
ϵ
(
0
°
-
β
,
0
°
+
β
)
, the first landscape/portrait direction is a portrait direction at
0
°
; When a
a
ϵ
(
90
°
-
β
,
90
°
+
β
)
, the first landscape/portrait direction is a portrait direction at
90
°
; When a
a
ϵ
(
180
°
-
β
,
180
°
+
β
)
, the first landscape/portrait direction is a portrait direction at
180
°
; and when a
a
ϵ
(
270
°
-
β
,
270
°
+
β
)
, the first landscape/portrait direction is a portrait direction at
270
°
, wherein
β
is greater than
0
°
and less than
45
°
.
However, it is obvious in the art that
0
°
,
90
°
,
180
°
and
270
°
correspond with cardinal directions, and in the context with a user orientation and natural direction of the device, relates to the angular difference between a direction of the device and a user’s orientation direction. For instance,
a
=
90
°
corresponds with a user orientation direction being 90 degrees rotated clockwise relative to a device. The invention of Gardiner pertains to the adjustment of a display direction based on a user’s orientation direction relative to the device orientation direction, and is demonstrated visually in Figure 1A, Figure 1B, and Figure 1C. While specific angular values
a
are not provided, it would have been obvious to a person having ordinary skill in the art that these angles relate to displaying content in each of the cardinal directions, which Gardiner teaches, where when the angle
a
ϵ
(
0
°
-
β
,
0
°
+
β
)
, the first landscape/portrait direction is a portrait direction at
0
°
; When
a
ϵ
(
90
°
-
β
,
90
°
+
β
)
, the first landscape/portrait direction is a portrait direction at
90
°
; When
a
ϵ
(
180
°
-
β
,
180
°
+
β
)
, the first landscape/portrait direction is a portrait direction at
180
°
; and when
a
ϵ
(
270
°
-
β
,
270
°
+
β
)
, the first landscape/portrait direction is a portrait direction at
270
°
.
Furthermore, while Gardiner does not explicitly state that
β
is greater than
0
°
and less than
45
°
, Gardiner does state “a threshold amount (e.g., a predetermined angle such as 45.degree”, where it is obvious in the art that the threshold for a direction recognition range may range between complete alignment of the user and a natural device orientation when the threshold is
0
°
and a maximum threshold of
45
°
, since a larger threshold would result in overlapping regions; a person having ordinary skill in the art would understand that any threshold within the specified range would perform the desired function with varying amounts of flexibility with regards to when the screen would switch its display orientation.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention that Gardiner, in combination with common knowledge of the art, implicitly teaches an angle
a
where
a
is representative of the cardinal directions in relation to the orientation of the user and a natural orientation of the device, and also implicitly teaches that a direction recognition range threshold
β
may be any value between
0
°
and
45
°
.
Regarding Claim 10, the method according to Claim 3 is rejected over Gardiner.
Gardiner teaches the method according to Claim 3, further comprising:
if the first landscape/portrait direction is a landscape direction, maintaining a current screen display direction of the electronic device, wherein the landscape direction comprises the landscape direction at
90
°
and the landscape direction at
270
°
(Paragraph [0016]: “The device's accelerometer or other integrated inertial sensor is monitored at 24 to determine when the device 10 has been tilted sufficiently to otherwise trigger a switch of the display mode from portrait to landscape or vice versa. This may be an entirely conventional operation. If a tilt is detected in such operation, an image of the user's face captured with the front-facing camera at that point is again analyzed at 26 to determine if the user's orientation has likewise changed. If the orientation of the device has tilted without a corresponding change in the user's orientation, then the display mode is switched at 28. On the other hand, if the user's orientation has also changed such that the user's face remains in approximately the same orientation relative to the display screen, then the current display mode is maintained at 30”. Notes: landscape directions conventionally correspond with
90
°
and
270
°
).
Regarding Claim 18, the method according to Claim 1 is rejected over Gardiner.
Gardiner teaches the method according to Claim 1, wherein the electronic device comprises:
a landscape/portrait direction calculation module and a face direction calculation module (Paragraph [0016]: “The device's accelerometer or other integrated inertial sensor is monitored at 24 to determine when the device 10 has been tilted sufficiently to otherwise trigger a switch of the display mode from portrait to landscape or vice versa. This may be an entirely conventional operation. If a tilt is detected in such operation, an image of the user's face captured with the front-facing camera at that point is again analyzed at 26 to determine if the user's orientation has likewise changed. If the orientation of the device has tilted without a corresponding change in the user's orientation, then the display mode is switched at 28. On the other hand, if the user's orientation has also changed such that the user's face remains in approximately the same orientation relative to the display screen, then the current display mode is maintained at 30. In other words, if the image feature of the user's face changes orientation with respect to the device's orientation by less than a threshold amount (e.g., a predetermined angle such as 45.degree.), then the switching of the display mode is disabled. Notes: a landscape/portrait direction calculation module and a face direction calculation module, in their broadest reasonable interpretations, is an entity that derives a landscape/portrait direction and a face direction. Therefore, the modules for said directions are inherent with the derivation of the directions); and
the obtaining, when the electronic device is in or enters a landscape side-standing state (Paragraph [0016]: “The device's accelerometer or other integrated inertial sensor is monitored at 24 to determine when the device 10 has been tilted sufficiently to otherwise trigger a switch of the display mode from portrait to landscape or vice versa. This may be an entirely conventional operation),
a first landscape/portrait direction of the electronic device determined based on a face direction comprises:
when the landscape/portrait direction calculation module determines that a posture of the electronic device is the landscape side-standing state, enabling the face direction calculation module, obtaining image data collected by a front camera, recognizing the face direction based on the image data, and determining the first landscape/portrait direction of the electronic device based on the face direction (Paragraph [0016]: “The device's accelerometer or other integrated inertial sensor is monitored at 24 to determine when the device 10 has been tilted sufficiently to otherwise trigger a switch of the display mode from portrait to landscape or vice versa. This may be an entirely conventional operation. If a tilt is detected in such operation, an image of the user's face captured with the front-facing camera at that point is again analyzed at 26 to determine if the user's orientation has likewise changed. If the orientation of the device has tilted without a corresponding change in the user's orientation, then the display mode is switched at 28. On the other hand, if the user's orientation has also changed such that the user's face remains in approximately the same orientation relative to the display screen, then the current display mode is maintained at 30).
Gardiner does not teach explicitly disabling the face direction calculation module.
However, Gardiner implicitly teaches disabling the face direction calculation module (Paragraph [0016]: “If a tilt is detected in such operation, an image of the user's face captured with the front-facing camera at that point is again analyzed at 26 to determine if the user's orientation has likewise changed”). Gardiner teaches that the face direction calculation module performs direction calculation when a tilt is detected; in other words, it is conditionally enabled when a tilt is detected. Therefore, the face direction calculation module is disabled when a tilt is not detected. A person having ordinary skill in the art would find it obvious that after a tilt and display direction is established, the face direction calculation module must be disabled, since the device is capable of establishing a screen direction resulting from tilting operations in real-time, and the face direction calculation module is only active after a tilting operation has been detected.
Therefore, it would have been obvious to a person having ordinary skill in the art that Gardiner implicitly teaches disabling the face direction calculation module.
Claims 4-5 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Gardiner (US 20120057064 A1) in view of Engman (US 20190329053 A1).
Regarding Claim 4, the method according to Claim 3 is rejected over Gardiner.
Gardiner does not teach When
a
ϵ
β
,
90
°
-
β
,
a
ϵ
90
°
+
β
,
180
°
-
β
,
a
ϵ
180
°
+
β
,
270
°
-
β
,
o
r
a
ϵ
270
°
+
β
,
360
°
-
β
, a value of the first landscape/portrait direction indicates unknown.
However, Engman teaches a rotation range for angle ranges, and when an angle falls into the angle range, a value of the first landscape/portrait direction indicates unknown (Paragraph [0032]: “the viewing angle may rotate when angle A1 or angle A2 ranges from 20 degrees to 60 degrees, 30 degrees to 50 degrees, or the like. A rotational range for angle A1 and angle A2 may prevent the screen from oscillating between various view orientations. For example, if the threshold was a strict degree threshold then the screen 128 may continuously alternate between viewpoints in an attempt to accommodate the shifting”. Notes: The broadest reasonable interpretation of indicating unknown in the context of the specification is a value that is non-actionable, as is indicated by a value defined by the range, which is described in the Applicant’s Specification, Paragraph [0022] as leading to the display direction not switching).
While Engman does not explicitly teach angle ranges that involve the specific values and
β
, it would have been obvious to a person having ordinary skill in the art that angle ranges can be defined according to a threshold value such as
β
, resulting in
a
ϵ
β
,
90
°
-
β
,
a
ϵ
90
°
+
β
,
180
°
-
β
,
a
ϵ
180
°
+
β
,
270
°
-
β
,
o
r
a
ϵ
270
°
+
β
,
360
°
-
β
, since Gardiner teaches the actionable display direction switching angle range specific to the establishment of the cardinal directions (
0
°
,
90
°
,
180
°
,
270
°
) and
β
(refer to the rejection of Claim 3) and Engman teaches establishing non-actionable angle range that results in a display direction not switching if the angle falls within the non-actionable angle range; It would have been obvious to a person having ordinary skill in the art that the angle ranges defined by Gardiner can be modified with the non-actionable angle ranges of Engman to result in less sensitive display switching.
Gardiner and Engman are considered analogous in the art with respect to the switching of a screen orientation with respect to a device orientation. One having ordinary skill in the art would be motivated to reduce constant screen orientation changes, which can result from a strict switching threshold from portrait to landscape and vice versa at a particular angle. As is evident in Engman, non-actionable angle ranges are utilized to alleviate this issue, resulting in less sensitive switching of a display direction.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the display direction switching of Gardiner with the non-actionable angle range of Engman; Doing so would yield the predictable result of reducing the sensitivity of the display direction switching to a more suitable level.
Regarding Claim 5, the method according to Claim 3 is rejected over Gardiner.
Gardiner teaches the method according to Claim 3, wherein a chip platform used by the electronic device performs direction recognition (Paragraph [0015]: “A process for implementing an embodiment of the present invention is shown in FIG. 2. The process can be performed by a suitably programmed processor in the device 10 (e.g., an applications processor)” Paragraph [0016]: “The device's accelerometer or other integrated inertial sensor is monitored at 24 to determine when the device 10 has been tilted sufficiently to otherwise trigger a switch of the display mode from portrait to landscape or vice versa. This may be an entirely conventional operation. If a tilt is detected in such operation, an image of the user's face captured with the front-facing camera at that point is again analyzed at 26 to determine if the user's orientation has likewise changed. If the orientation of the device has tilted without a corresponding change in the user's orientation, then the display mode is switched at 28. On the other hand, if the user's orientation has also changed such that the user's face remains in approximately the same orientation relative to the display screen, then the current display mode is maintained at 30. In other words, if the image feature of the user's face changes orientation with respect to the device's orientation by less than a threshold amount (e.g., a predetermined angle such as 45.degree.), then the switching of the display mode is disabled”. Notes: a chip platform, in its broadest reasonable interpretation, is a processing chip (processor)).
Gardiner does not explicitly teach that the angular resolution that direction recognition is performed at is
3
0
°
, and that
β
=
30
°
, although it is implicit that the angular resolution and
β
can be 45 degrees.
However, Engman teaches an angular resolution that direction recognition is performed at is
30
°
, and that
β
=
30
°
(Paragraph [0008]: “In some embodiments, the processor is further configured to determine when the motion passes an angular threshold from a baseline plane of the user interface. In some embodiments, the angular threshold is between about 30 and about 60 degrees from the baseline plane”. Notes: The broadest reasonable interpretation of angular resolution is the degree to which a device can perform direction recognition. Because Engman’s device is configured to determine when the angle passes a threshold, where the threshold is stated as being capable of being between 30 and 60 degrees, Engman’s device has an angular resolution of 30 degrees).
Gardiner and Engman are considered analogous in the art with respect to the directional display of a screen with respect to a device’s orientation. A common motivation in the art is to establish a threshold value for determining a specific angle range, as is evident in both Gardiner and Engman. The threshold defines angle ranges with respect to established key directions, from which screen orientation can be based upon.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the direction recognition and screen direction adjustment according to a user and device orientation of Gardiner with the angular resolution and
β
being 30 degrees for a screen direction adjustment device of Engman; Doing so would yield the predictable result of the operation of a screen adjustment device according to a user and device orientation being centered the angular resolution and
β
being 30 degrees.
Regarding Claim 9, the method according to Claim 4 is rejected over Gardiner as modified.
Gardiner as modified teaches the method according to Claim 4, further comprising:
If the value of the first landscape/portrait direction indicates unknown, maintaining a current screen display direction of the electronic device (Engman, Paragraph [0032]: “the viewing angle may rotate when angle A1 or angle A2 ranges from 20 degrees to 60 degrees, 30 degrees to 50 degrees, or the like. A rotational range for angle A1 and angle A2 may prevent the screen from oscillating between various view orientations. For example, if the threshold was a strict degree threshold then the screen 128 may continuously alternate between viewpoints in an attempt to accommodate the shifting”. Notes: The broadest reasonable interpretation of indicating unknown in the context of the specification is a value that is non-actionable, as is indicated by a value defined by the range, which is described in the Applicant’s Specification, Paragraph [0022] as leading to the display direction not switching”).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Gardiner (US 20120057064 A1) in view of Kadric (Using YOLO for Object Detection: How to Extract People Images, January 2023).
Regarding Claim 8, the method according to Claim 7 is rejected over Gardiner.
Gardiner does not teach that if no face is recognized or a plurality of faces are recognized based on the image data, or there is an abnormality in recognition, determining that a value of the first landscape/portrait direction indicates unknown.
However, Kadric teaches recognizing a plurality of faces (“After running this code, you should be able to see an image with bounding boxes around any people that were detected. Check the image below”).
Gardiner and Kadric are considered analogous in the art with respect to identifying people’s faces in images. One would be motivated to identify multiple faces within an image when different tasks are performed based on the presence of multiple faces, as is evident in Gardiner, where the orientation of a screen is based upon the orientation of a single user’s face.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the facial recognition of Gardiner with the recognition of multiple people and faces of Kadric; Doing so would yield the predictable result of customizing the display direction of a screen to only a single person to prevent confusion when deciding on a person to orient the display around.
Gardiner as modified does not explicitly teach that as a result of recognizing a plurality of faces, a value of the first landscape/portrait direction indicates unknown.
However, it would have been obvious to a person having ordinary skill in the art that because Gardiner as modified depends on the recognition of a single user face to determine user orientation with respect to a device’s orientation (Gardiner, Paragraph [0016]: “Multiple images of the user, which may be captured by the same camera 14 as the device 10 is tilted from one orientation to another (e.g., horizontal to vertical or vice versa), may be analyzed to make the determination as to how much the user's feature changes its orientation”; Gardiner, Paragraph [0015]: “the iPhoto.TM. application by Apple Inc. uses facial detection to identify faces of people in photographs”), having multiple faces within an image captured by the camera of the device could cause problems with determining which face orientation to use; hence in the case that there is a plurality of faces recognized based on the image data, determining that a value of the first landscape/portrait direction indicates unknown (Notes: a value indicating unknown, in its broadest reasonable interpretation, is indicating that a change in display direction does not occur, as is noted by the Applicant’s Specification in Paragraph [0022].) is obvious.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to indicate an unknown value, and subsequently prevent the screen display direction from switching, in a case where there are multiple faces present in an image captured by the device of Gardiner as modified.
Claim 14 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Gardiner (US 20120057064 A1) in view of Michelson (US 20110257535 A1) and Park (US 20150365566 A1).
Regarding Claim 14, the method according to Claim 13 is rejected over Gardiner.
Gardiner teaches the method according to Claim 13, further comprising:
When the electronic device exits the landscape side-standing state, or when there is no landscape/portrait auto-rotation requirement in the current scenario, conducting a sampling manner of obtaining the image data collected by the front camera in a real-time sampling manner (Paragraph [0016]: “The device's accelerometer or other integrated inertial sensor is monitored at 24 to determine when the device 10 has been tilted sufficiently to otherwise trigger a switch of the display mode from portrait to landscape or vice versa. If a tilt is detected in such operation, an image of the user's face captured with the front-facing camera at that point is again analyzed at 26 to determine if the user's orientation has likewise changed. If the orientation of the device has tilted without a corresponding change in the user's orientation, then the display mode is switched at 28. On the other hand, if the user's orientation has also changed such that the user's face remains in approximately the same orientation relative to the display screen, then the current display mode is maintained at 30”).
Gardiner does not teach an interval sampling manner.
However, Park teaches an interval sampling manner (Paragraph [0005]: “There is a tradeoff between the time for image processing and the detection accuracy of lesions detected by image processing”; Paragraph [0006]: “In the conventional real time CAD technology this tradeoff is balanced such that CAD processing is performed only on selected images rather than all of the captured images. Images subjected to the CAD processing are selected by sampling at equal time intervals such that the images equally include captured human body regions”).
Gardiner and Park are considered analogous in the art with respect to the sampling and processing of images. One would be motivated to perform interval sampling instead of real-time sampling to reduce the increase in time for image processing, as is evident in Park.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the electronic device of Gardiner with the interval sampling manner of Park; doing so would yield the predictable result of reducing the time for image processing when determining whether a display direction should be switched.
Gardiner as modified does not teach switching a sampling manner to an interval sampling manner from a real-time sampling manner.
However, Michelson teaches switching between a real-time sampling manner and interval sampling manner (Paragraph [0035]: “Whether implemented as in FIG. 1B or otherwise, the processing of the signals, and/or of additional information received by processor 220, may be carried out at different times, according to different embodiments of the invention. Also, different acts of processing (possibly of different processing levels) may be carried out at different times. Such times may be, for example, a continuous real time, or near-real time processing, timed processing at predetermined intervals, processing when processor 220 is relatively free from other processing tasks, processing after the monitoring period is over, and so forth. It is noted that in different situations (which may be related to exceeding of the first and/or the second thresholds), processor 220 may determine to transmit at least a portion of the information gathered to a remote entity (e.g. a control center, over cellular telephony network), for more extensive processing”. Notes: the broadest reasonable interpretation of sampling, in the context of the invention, is processing data obtained in real-time).
Gardiner as modified and Michelson are considered analogous in the art with respect to the processing of information from sensors. One would be motivated to swap between real-time sampling and interval sampling for a device depending on whether the scenario and needs, particularly related to the processing capability of the device.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the real-time sampling and interval sampling manner of Gardiner as modified with the ability to switch between the two sampling manners of Michelson; Doing so would yield the predictable result of optimizing processing of the images when determining whether a display direction needs to be switched.
Regarding Claim 16, the method according to Claim 14 is rejected over Gardiner as modified.
Gardiner as modified teaches the method according to Claim 14, after the switching a sampling manner of obtaining the image data collected by the front camera from the real-time sampling manner to an interval sampling manner, further comprising:
The electronic device enters or is in the landscape side-standing state and there is the land scape/portrait auto-rotation requirement in the current scenario (Gardiner, Paragraph [0016]: “The device's accelerometer or other integrated inertial sensor is monitored at 24 to determine when the device 10 has been tilted sufficiently to otherwise trigger a switch of the display mode from portrait to landscape or vice versa. This may be an entirely conventional operation”; Refer to Gardiner, Figure 1A, 1B, and 1C);
And switching the sampling manner of obtaining the image data collected by the front camera from the interval sampling manner to the real-time sampling manner (Michelson, Paragraph [0035]: “Whether implemented as in FIG. 1B or otherwise, the processing of the signals, and/or of additional information received by processor 220, may be carried out at different times, according to different embodiments of the invention. Also, different acts of processing (possibly of different processing levels) may be carried out at different times. Such times may be, for example, a continuous real time, or near-real time processing, timed processing at predetermined intervals, processing when processor 220 is relatively free from other processing tasks, processing after the monitoring period is over, and so forth. It is noted that in different situations (which may be related to exceeding of the first and/or the second thresholds), processor 220 may determine to transmit at least a portion of the information gathered to a remote entity (e.g. a control center, over cellular telephony network), for more extensive processing”; Park, Paragraph [0006]: “In the conventional real time CAD technology this tradeoff is balanced such that CAD processing is performed only on selected images rather than all of the captured images. Images subjected to the CAD processing are selected by sampling at equal time intervals such that the images equally include captured human body regions”. Notes: Refer to the combination rationale of Claim 14).
Gardiner as modified does not teach when the electronic device enters or is in the landscape side-standing state and there is the land scape/portrait auto-rotation requirement in the current scenario, immediately switching the sampling manner of obtaining the image data collected by the front camera from the interval sampling manner to the real-time sampling manner.
However, it would have been obvious to a person having ordinary skill in the art that when the electronic device enters or is in the landscape side-standing state and there is the land scape/portrait auto-rotation requirement in the current scenario, immediately switching the sampling manner of obtaining the image data collected by the front camera from the interval sampling manner to the real-time sampling manner. The display direction of Gardiner’s electronic device is derived dynamically from the orientation of a user with respect to the orientation of the device (Gardiner, Paragraph [0016]: “The device's accelerometer or other integrated inertial sensor is monitored at 24 to determine when the device 10 has been tilted sufficiently to otherwise trigger a switch of the display mode from portrait to landscape or vice versa. If a tilt is detected in such operation, an image of the user's face captured with the front-facing camera at that point is again analyzed at 26 to determine if the user's orientation has likewise changed. If the orientation of the device has tilted without a corresponding change in the user's orientation, then the display mode is switched at 28. On the other hand, if the user's orientation has also changed such that the user's face remains in approximately the same orientation relative to the display screen, then the current display mode is maintained at 30”). When the device is in an established position, there is not a need for real-time sampling of image data, and interval sampling is sufficient; interval sampling reduces the processing required in the background. However, in a case where a tilt of the device is detected such that it is in landscape mode, verifying the user’s orientation becomes necessary for establishing a display direction. In this scenario, it would be obvious to a person having ordinary skill in the art that a new position of the device should result in real-time sampling of image data to establish a user orientation to quickly determine whether the display direction should be changed.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date that a change in the orientation of a device capable of switching between interval sampling and real-time sampling of image data should switch to real-time sampling when a new device orientation is established; Doing so would yield the predictable result of enabling the determination of whether a display direction should be switched depending on the user orientation promptly.
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Gardiner (US 20120057064 A1) in view of Michelson (US 20110257535 A1) and Park (US 20150365566 A1), in further view of Aratani (US 20200104969 A1).
Regarding Claim 15, the method according to Claim 14 is rejected over Gardiner as modified.
Gardiner as modified does not teach that when it is determined that the sampling manner of obtaining the image data collected by the front camera needs to be switched from real-time sampling to interval sampling, obtaining delay duration; and when timing reaches the delay duration switching the sampling manner of obtaining the image data collected by the front camera from the real-time sampling manner to the interval sampling manner.
However, Aratani teaches determining that the sampling manner of obtaining the image data collected by the camera needs to be switched from real-time sampling to interval sampling, and obtaining delay duration (Paragraph [0083]: “A load status of a frame CPU in the frame t at the time when the current image is obtained is expressed by the CPU utilization ratio Qt. In a case where the CPU utilization ratio Q.sub.t is greater than the threshold value Q.sub.Th, the processing takes a time since a loading factor of the CPU is high, and there is a possibility that a processing time per frame cannot catch up with an update interval of image capturing”; Paragraph [0084: “As described above, the CPU load status can be used as the mode switching information. The mode is determined to be the frame rate priority mode or to be the estimation accuracy priority mode according to the CPU load status”);
And when timing reaches the delay duration, switching the sampling manner of obtaining the image data collected by the camera from the real-time sampling manner to the interval sampling manner (Paragraph [0084: “As described above, the CPU load status can be used as the mode switching information. The mode is determined to be the frame rate priority mode or to be the estimation accuracy priority mode according to the CPU load status”).
Gardiner as modified and Aratani are considered analogous in the art with respect to processing of obtained images. A common motivation is to utilize a threshold duration for determining whether to swap between an interval sampling rate and real-time sampling rate, as is evident in Aratani.
Therefore, it would have been obvious to a person having ordinary skill in the art to combine the ability to swap between interval sampling and real-time sampling of images of Gardiner as modified with the delay duration threshold of Aratani for prompting a swap between interval sampling and real-time sampling of images; Doing so would yield the predictable result of establishing an efficient system for sampling images when considering the processing time required for the obtained images.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RAYMOND CHUN LAM LI whose telephone number is (571)272-5124. The examiner can normally be reached M-F 8:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at 571-272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RAYMOND CHUN LAM LI/Examiner, Art Unit 2614
/KENT W CHANG/Supervisory Patent Examiner, Art Unit 2614
Prosecution Insights